Automating cluster creation and management for Apache Spark in Openstack cloud

Approximate Stream Analytics in Apache Flink and Apache Spark Streaming

Approximate computing aims for efficient execution of workflows where an approximate output is sufficient instead of the exact output. The idea behind approximate computing is to compute over a representative sample instead of the entire input dataset. Thus, approximate computing — based on the chosen sample size — can make a systematic trade-off between the output accuracy and computation effi...

Getting Started with Apache Spark

Automating Cloud Application Management Using Management Idioms

Patterns are a well-established concept to document generic solutions for recurring problems in an abstract manner. Especially in Information Technology (IT), many pattern languages exist that ease creating application architectures, designs, and management processes. Their generic nature provides a powerful means to describe knowledge in an abstract fashion that can be reused and refined for c...

OCReM: OpenStack-based cloud datacentre resource monitoring and management scheme

Managing virtualised computing, network and storage resources at large-scale in both public and private cloud datacentres is a challenging task. As an open source cloud operating system, OpenStack needs to be enhanced for managing cloud datacentre resources. In order to improve OpenStack functions to support cloud datacentre resource management, we present OCReM: OpenStack-based cloud datacentr...

MLlib: Machine Learning in Apache Spark

Apache Spark is a popular open-source platform for large-scale data processing that is well-suited for iterative machine learning tasks. In this paper we present MLlib, Spark’s open-source distributed machine learning library. MLlib provides efficient functionality for a wide range of learning settings and includes several underlying statistical, optimization, and linear algebra primitives. Shi...